In fields of computing, communications, data storage, and the like, information may be transferred from a system memory associated with one computer system to a system memory associated with another computer system. Communication between computer systems may involve exchanging and processing messages through a proprietary protocol stack at each of the computer systems.
The Remote Direct Memory Access Consortium (RDMAC) has specified a set of verbs that enable an operating system to supply an infrastructure that supports Remote Direct Memory Access (RDMA) over Internet Wide Area RDMA Protocol (iWARP) technology using Queue Pair (QP) semantics. The infrastructure may include a flow control protocol for performing data transfer operations. The RDMA-capable protocol may be constructed in a manner that is not cognizant of the underlying fabric infrastructure, for example iWARP. Windows Sockets Direct (WSD) and Sockets Direct Protocol (SDP) are two example protocols that support RDMA semantics over a variety of interconnect fabrics and can use QP semantics.
WSD and SDP are connection-oriented protocols. Once a connection is established, data transfer can be performed using RDMA transfers, characterized as application buffer to application buffer transfers, or non-RDMA transfers, which are protocol buffer to protocol buffer transfers. Each end of a connection that uses QP semantics includes a QP composed of a Send Queue and a Receive Queue. Each send operation, including one or more protocol buffers, is posted on the Send Queue and each receive operation, including one or more protocol buffers, is posted on the Receive Queue of that connection. Operations are processed in the sequence queued. Each queued operation can choose to take a completion. The completions are queued to a Completion Queue (CQ) associated with the QP upon processing of the queued operation.